Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.

Documentation preview: https://vllm--30566.org.readthedocs.build/en/30566/

I believe we took care of all the CI failures with transformers v5 upgrade. Thanks @bigPYJ1151 for the CPU fix!

Running full CI again now: https://buildkite.com/vllm/ci/builds/61345 hopefully it's the last

Claude's approach for basics model test (extra init 2)

▎ Skip XverseForCausalLM tests on transformers v5
▎
▎ The XVERSE tokenizer (xverse/XVERSE-7B-Chat) is incompatible with transformers v5 —
▎ AutoTokenizer.from_pretrained fails with add_prefix_space does not match declared prepend_scheme
▎ due to a mismatch in the model's tokenizer.json. This is an upstream issue in the XVERSE tokenizer
▎ files, not in vLLM or transformers.
▎
▎ Added max_transformers_version="4.57" with transformers_version_reason={"vllm": ...} so both
▎ test_registry_imports and test_can_initialize_large_subset skip this model on transformers v5.

Claude's fix for this test: https://buildkite.com/vllm/ci/builds/61345#019d8f38-a3e5-47f5-94aa-031e3b466e29/L3122

test_olmoe_lora — NameError: _get_lora_aux_cuda_stream is not defined (cb03f5d)

The _get_lora_aux_cuda_stream function, lora_linear_async, and the direct_register_custom_op call
were all inside an if envs.VLLM_LORA_ENABLE_DUAL_STREAM: block evaluated at import time. The test
fixture sets the env var via monkeypatch.setenv after import, so the names were never defined when
the runtime code tried to use them. Moved all definitions outside the conditional — they're only
invoked when _enable_aux_cuda_stream is True (checked at runtime in _init_lora_stream_context and
apply), so registering them unconditionally is safe.

Claude's fix for step3 tool parser

2. test_step3_tool_parser — spaces stripped in streaming tests (e187e72)

Transformers v5's LlamaTokenizerFast.init unconditionally replaces the pre-tokenizer from
tokenizer.json with Metaspace. For models like stepfun-ai/step3 whose tokenizer uses ByteLevel, this
causes spaces to be silently dropped. Added _restore_original_pretokenizer() in vllm/tokenizers/hf.py
that detects the mismatch and restores the original pre-tokenizer/decoder from tokenizer.json.

3. test_minimax_tool_parser — trust_remote_code error (e187e72)

Transformers v5 now calls AutoConfig.from_pretrained internally during tokenizer loading. For
custom-code models like MiniMax, this requires trust_remote_code=True. Added the missing parameter to
the test fixture.

Claude's fix for https://buildkite.com/vllm/ci/builds/61345#019d8f38-789f-401f-b021-183d4141b2f8/L2600

Fix test_quant_model_lora crash — LoRA adapter path passed to get_config() (cc19a1b)

The get_config() call added in get_tokenizer() (commit 8f551d0) pre-registers custom vllm configs
with AutoConfig before tokenizer loading. However, tokenizer_name can be a LoRA adapter directory
(e.g. jashing/tinyllama-colorist-lora), which doesn't contain a config.json — only
adapter_config.json. This causes get_config() to raise ValueError: Invalid repository ID or local
directory specified.

Wrapped the call with contextlib.suppress(ValueError, OSError) since the config pre-registration is
only relevant for models with custom vllm configs, not for LoRA adapter or tokenizer-only paths.

For step3_vl, the issue is that tokenizer_config.json explicitly sets the wrong tokenizer_class (it is not a LlamaTokenizerFast).

I've pushed 816db8b which uses TokenizersBackend instead. This class will determing what the tokenizer is based on the actual tokenizer.json and is significantly more reliable. We can use this mechanism for other checkpoints we find with the same issue.

The longer term solution is to upstream this override to Transformers, which I have done in huggingface/transformers#45449.

I've moved the fix for VLLM_LORA_ENABLE_DUAL_STREAM to the test code so that the behaviour in vLLM is unchanged

full CI run: https://buildkite.com/vllm/ci/builds/61509
hopefully we can close this at exactly 200 commits xD